How Useful is Self-Supervised Pretraining for Visual Tasks?

Recent advances have spurred incredible progress in self-supervised pretraining for vision. We investigate what factors may play a role in the utility of these pretraining methods for practitioners. To do this, we evaluate various self-supervised algorithms across a comprehensive array of synthetic datasets and downstream tasks. We prepare a suite of synthetic data that enables an endless supply of annotated images as well as full control over dataset difficulty. Our experiments offer insights into how the utility of self-supervision changes as the number of available labels grows as well as how the utility changes as a function of the downstream task and the properties of the training data. We also find that linear evaluation does not correlate with finetuning performance. Code and data is available at \href{https://www.github.com/princeton-vl/selfstudy}{github.com/princeton-vl/selfstudy}.Comment: To appear in CVPR 202

Exploiting temporal information for 3D pose estimation

In this work, we address the problem of 3D human pose estimation from a sequence of 2D human poses. Although the recent success of deep networks has led many state-of-the-art methods for 3D pose estimation to train deep networks end-to-end to predict from images directly, the top-performing approaches have shown the effectiveness of dividing the task of 3D pose estimation into two steps: using a state-of-the-art 2D pose estimator to estimate the 2D pose from images and then mapping them into 3D space. They also showed that a low-dimensional representation like 2D locations of a set of joints can be discriminative enough to estimate 3D pose with high accuracy. However, estimation of 3D pose for individual frames leads to temporally incoherent estimates due to independent error in each frame causing jitter. Therefore, in this work we utilize the temporal information across a sequence of 2D joint locations to estimate a sequence of 3D poses. We designed a sequence-to-sequence network composed of layer-normalized LSTM units with shortcut connections connecting the input to the output on the decoder side and imposed temporal smoothness constraint during training. We found that the knowledge of temporal consistency improves the best reported result on Human3.6M dataset by approximately $12.2\%$ and helps our network to recover temporally consistent 3D poses over a sequence of images even when the 2D pose detector fails

Alleviating Human-level Shift : A Robust Domain Adaptation Method for Multi-person Pose Estimation

Human pose estimation has been widely studied with much focus on supervised learning requiring sufficient annotations. However, in real applications, a pretrained pose estimation model usually need be adapted to a novel domain with no labels or sparse labels. Such domain adaptation for 2D pose estimation hasn't been explored. The main reason is that a pose, by nature, has typical topological structure and needs fine-grained features in local keypoints. While existing adaptation methods do not consider topological structure of object-of-interest and they align the whole images coarsely. Therefore, we propose a novel domain adaptation method for multi-person pose estimation to conduct the human-level topological structure alignment and fine-grained feature alignment. Our method consists of three modules: Cross-Attentive Feature Alignment (CAFA), Intra-domain Structure Adaptation (ISA) and Inter-domain Human-Topology Alignment (IHTA) module. The CAFA adopts a bidirectional spatial attention module (BSAM)that focuses on fine-grained local feature correlation between two humans to adaptively aggregate consistent features for adaptation. We adopt ISA only in semi-supervised domain adaptation (SSDA) to exploit the corresponding keypoint semantic relationship for reducing the intra-domain bias. Most importantly, we propose an IHTA to learn more domain-invariant human topological representation for reducing the inter-domain discrepancy. We model the human topological structure via the graph convolution network (GCN), by passing messages on which, high-order relations can be considered. This structure preserving alignment based on GCN is beneficial to the occluded or extreme pose inference. Extensive experiments are conducted on two popular benchmarks and results demonstrate the competency of our method compared with existing supervised approaches.Comment: Accepted By ACM MM'202

Ice Up, cubos de hielo saborizados

Objetivo: Investigar e implementar un proyecto de negocio sostenible de cubos de hielo saborizados para personas de 18 a 24 años de nivel socioeconómico A,B,C y D que vivan en Lima. Este proyecto es un negocio con fines de lucro, por lo cual se ha recibido dinero por parte de los clientes como una muestra que el proyecto ha tenido un resultado satisfactorio y se encuentra en buen camino para ponerse en marcha; Método: Este estudio ha tenido un método cualitativo y cuantitativo, esto buscando conocer cuánta aceptación podría tener nuestro producto con nuestro público objetivo buscando sostener mediante estos experimentos nuestro proyecto. Además, se pudieron realizar entrevistas al público objetivo de nuestro producto, asimismo buscamos conocer el tamaño de mercado de nuestros productos. Por último, realizamos una proyección de ventas de nuestras 4 presentaciones a lo largo de 3 años, buscando conocer si nuestro producto a largo plazo sería rentable y nos permitirá obtener ingresos que permitan mantener estable el negocio; Resultados: Luego de realizar los experimentos y el concierge, validamos nuestras hipótesis y logramos vender nuestros cubos de hielo saborizados; Conclusiones: Podemos concluir que los temas en el curso nos permitieron tener una gran guía, la cual nos sirvió de gran ayuda para poder realizar este proyecto, nuestro grupo de trabajo ha estado comprometido desde el inicio del proyecto, y logramos conectar con los clientes, con quienes estamos seguros continuaremos en contacto luego del fin de este cursoObjective: Investigate and implement a sustainable business project for flavored ice cubes for people between 18 and 24 years old of socioeconomic level A, B, C and D who live in Lima. This project is a for-profit business, for which money has been received from clients as a sign that the project has had a satisfactory result and is well on its way to start up; Method: This study has had a qualitative and quantitative method, this seeking to know how much acceptance our product could have with our target audience seeking to sustain our project through these experiments. In addition, interviews were conducted with the target audience of our product, we also sought to know the market size of our products. Finally, we carry out a sales projection of our 4 presentations over 3 years, seeking to know if our product in the long term would be profitable and will allow us to obtain income that allows us to keep the business stable; Results: After conducting the experiments and the concierge, we validated our hypotheses and were able to sell our flavored ice cubes; Conclusions: We can conclude that the topics in the course allowed us to have a great guide, which was of great help to us to carry out this project, our work group has been committed since the beginning of the project, and we managed to connect with clients, with whom we are sure we will continue in contact after the end of this courseTrabajo de investigació

Imaging-based clusters in current smokers of the COPD cohort associate with clinical characteristics: the SubPopulations and Intermediate Outcome Measures in COPD Study (SPIROMICS)

Background Classification of COPD is usually based on the severity of airflow, which may not sensitively differentiate subpopulations. Using a multiscale imaging-based cluster analysis (MICA), we aim to identify subpopulations for current smokers with COPD. Methods Among the SPIROMICS subjects, we analyzed computed tomography images at total lung capacity (TLC) and residual volume (RV) of 284 current smokers. Functional variables were derived from registration of TLC and RV images, e.g. functional small airways disease (fSAD%). Structural variables were assessed at TLC images, e.g. emphysema and airway wall thickness and diameter. We employed an unsupervised method for clustering. Results Four clusters were identified. Cluster 1 had relatively normal airway structures; Cluster 2 had an increase of fSAD% and wall thickness; Cluster 3 exhibited a further increase of fSAD% but a decrease of wall thickness and airway diameter; Cluster 4 had a significant increase of fSAD% and emphysema. Clinically, Cluster 1 showed normal FEV1/FVC and low exacerbations. Cluster 4 showed relatively low FEV1/FVC and high exacerbations. While Cluster 2 and Cluster 3 showed similar exacerbations, Cluster 2 had the highest BMI among all clusters. Conclusions Association of imaging-based clusters with existing clinical metrics suggests the sensitivity of MICA in differentiating subpopulations

Imaging-based clusters in former smokers of the COPD cohort associate with clinical characteristics: the SubPopulations and intermediate outcome measures in COPD study (SPIROMICS)

Background: Quantitative computed tomographic (QCT) imaging-based metrics enable to quantify smoking induced disease alterations and to identify imaging-based clusters for current smokers. We aimed to derive clinically meaningful sub-groups of former smokers using dimensional reduction and clustering methods to develop a new way of COPD phenotyping. Methods: An imaging-based cluster analysis was performed for 406 former smokers with a comprehensive set of imaging metrics including 75 imaging-based metrics. They consisted of structural and functional variables at 10 segmental and 5 lobar locations. The structural variables included lung shape, branching angle, airway-circularity, airway-wall-thickness, airway diameter; the functional variables included regional ventilation, emphysema percentage, functional small airway disease percentage, Jacobian (volume change), anisotropic deformation index (directional preference in volume change), and tissue fractions at inspiration and expiration. Results: We derived four distinct imaging-based clusters as possible phenotypes with the sizes of 100, 80, 141, and 85, respectively. Cluster 1 subjects were asymptomatic and showed relatively normal airway structure and lung function except airway wall thickening and moderate emphysema. Cluster 2 subjects populated with obese females showed an increase of tissue fraction at inspiration, minimal emphysema, and the lowest progression rate of emphysema. Cluster 3 subjects populated with older males showed small airway narrowing and a decreased tissue fraction at expiration, both indicating air-trapping. Cluster 4 subjects populated with lean males were likely to be severe COPD subjects showing the highest progression rate of emphysema. Conclusions: QCT imaging-based metrics for former smokers allow for the derivation of statistically stable clusters associated with unique clinical characteristics. This approach helps better categorization of COPD sub-populations; suggesting possible quantitative structural and functional phenotypes.NIH [U01-HL114494, R01-HL112986, S10-RR022421]; Basic Science Research Program through the National Research Foundation of Korea (NRF) - Ministry of Education [NRF-2017R1D1A1B03034157]; Korea Ministry of Environment (MOE) [RE201806039]; NIH/NHLBI [HHSN268200900013C, HHSN268200900014C, HHSN268200900015C, HHSN268200900016C, HHSN268200900017C, HHSN268200900018C, HHSN268200900019C, HHSN268200900020C]Open access journalThis item from the UA Faculty Publications collection is made available by the University of Arizona with support from the University of Arizona Libraries. If you have questions, please contact us at [email protected]

Soluble receptor for advanced glycation end products (sRAGE) as a biomarker of COPD

BACKGROUND: Soluble receptor for advanced glycation end products (sRAGE) is a proposed emphysema and airflow obstruction biomarker; however, previous publications have shown inconsistent associations and only one study has investigate the association between sRAGE and emphysema. No cohorts have examined the association between sRAGE and progressive decline of lung function. There have also been no evaluation of assay compatibility, receiver operating characteristics, and little examination of the effect of genetic variability in non-white population. This manuscript addresses these deficiencies and introduces novel data from Pittsburgh COPD SCCOR and as well as novel work on airflow obstruction. A meta-analysis is used to quantify sRAGE associations with clinical phenotypes. METHODS: sRAGE was measured in four independent longitudinal cohorts on different analytic assays: COPDGene (n = 1443); SPIROMICS (n = 1623); ECLIPSE (n = 2349); Pittsburgh COPD SCCOR (n = 399). We constructed adjusted linear mixed models to determine associations of sRAGE with baseline and follow up forced expiratory volume at one second (FEV1) and emphysema by quantitative high-resolution CT lung density at the 15th percentile (adjusted for total lung capacity). RESULTS: Lower plasma or serum sRAGE values were associated with a COPD diagnosis (P < 0.001), reduced FEV1 (P < 0.001), and emphysema severity (P < 0.001). In an inverse-variance weighted meta-analysis, one SD lower log10-transformed sRAGE was associated with 105 ± 22 mL lower FEV1 and 4.14 ± 0.55 g/L lower adjusted lung density. After adjusting for covariates, lower sRAGE at baseline was associated with greater FEV1 decline and emphysema progression only in the ECLIPSE cohort. Non-Hispanic white subjects carrying the rs2070600 minor allele (A) and non-Hispanic African Americans carrying the rs2071288 minor allele (A) had lower sRAGE measurements compare to those with the major allele, but their emphysema-sRAGE regression slopes were similar. CONCLUSIONS: Lower blood sRAGE is associated with more severe airflow obstruction and emphysema, but associations with progression are inconsistent in the cohorts analyzed. In these cohorts, genotype influenced sRAGE measurements and strengthened variance modelling. Thus, genotype should be included in sRAGE evaluations